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TITLE OF THE INVENTION 

[0001] SYSTEM AND METHOD FOR ENHANCING A SERIAL 

TRANSMISSION 

FIELD OF THE INVENTION 

[0002] The present invention relates to a system and method for enhancing 
a serial transmission. More specifically, the present invention is related to a 
system and method for formatting and merging collected data with a television 
signal to yield an enhanced signal. 

BACKGROUND OF THE INVENTION 

[0003] The migration of present day television broadcasting more and more 
towards the use of the digital form as the preferred method of transmission, 
combined with the digitization of audio transmission and the ever increasing use 
of the Internet as a preferred form of data transfer, has led to a homogenization 
of the underlying media used for the transmission of all these sources of data. 

[0004] The above migration, often referred to as convergence of the 
transmission media, has not only affected the means by which information is 
transmitted, but is also having a major impact on the manner in which information 
is used and the manner in which a viewer interacts with that information. 
Additionally, the increasing penetration of the combination of the general purpose 
Personal Computer (PC) and the Internet into North American households, as well 
as the marked increase in the amount and types of information which may be 
transmitted by the data networks underlying the Internet, appears to herald the 
end of the traditional method of TV broadcasting, whereby a viewer passively 
receives information, with little influence on the information he receives or the 
manner in which that information is displayed. 
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[0005] One major advantage digital transmission methods have over 
traditional analogue broadcasting methods is the inherent two-way nature of the 
information flow. Although broadcasting in its current form, as discussed above, 
looks set to continue well on into the future, the two-way nature of the 
transmission media has opened up exciting new avenues of development for the 
producers of content slated for broadcast. 

[0006] One aspect which is currently the object of much interest is the use 
of two-way transmission to enhance a broadcast viewing experience by providing 
the viewer with the tools to interact with a broadcast and customize the manner 
in which the transmitted information is displayed. Traditionally live television 
broadcasts are the result of multiple video and audio feeds and a producer's talent 
for combining the feeds to create content which provides for maximum visual 
effect while maintaining viewer attention. 

[0007] Recent developments in image processing, transmission and 
computerized support for image capture and editing have provided the producer 
with an array of tools with which he can further enhance the content of a 
broadcast. One important aspect is the combination of animated graphics with the 
video being broadcast, primarily through superimposing the graphics on the 
broadcast video, as a method for conveying to the viewer additional detailed 
information on a given aspect of the broadcast without interrupting the 
transmission thereof. 

[0008] A number of systems and approaches have been disclosed in the 
following prior art documents which take advantage of the two-way nature of 
communications to provide some degree of interactivity between viewer and 
broadcaster as well as the manner in which information is displayed: 
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Freeman et al. 



[0009] 



United States patent No.5,534,913 by Majeti et al. discloses an 



apparatus which uses an existing broadcasting network, such as a cable 
television distribution network, to transfer data to viewers equipped with personal 
computers, while simultaneously allowing such viewers to transmit data from their 
computers to the enhanced data providers via a secondary communication path 
such as that provided by modem and the public telephone system. The patent 
describes primarily a system for using a portion of the bandwidth of an existing 
cable network as the downlink communication path for a generalized packet data 
communication system. The preferred embodiment, however, does mention the 
use of the system for a number of enhanced services, e.g. the display of a 
message indicating that any of a number of predetermined conditions have arisen 
(the example of when the price of a given stock reaches a viewer defined target 
value is provided). The message is displayed on a portion of the screen of the TV 
(1/3 rd of the screen is proposed) so as not to interrupt the normal viewing. 

[0010] United States patent No.5,585,858 by Harper et al. describes a 
system for simultaneously broadcasting an interactive TV program with a normal 
conventional broadcast in the same standard video signal bandwidth. The 
interactive portion of the programming consists of audio and graphic data, a 
portion of which may be stored locally on a local storage device such as a CD 
ROM or solid state game cartridge. The viewer can individualize a particular 
broadcast by selecting audio and video graphic overlays. This document also 
discloses the use of "trigger points" spread throughout a broadcast which, when 
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activated, invoke macros which call for certain interactive function, e.g. the overlay 
of a particular graphics screen or the playing of a personalized audio signal. The 
exact timing of the trigger points is unknown to the viewer in order to make them 
appear unsolicited. 

[0011] United States patent No.5,774,664 by Hidary et al. discloses a 
system for integrating video with data available over the Internet. In the preferred 
embodiment, this is done by coding a Uniform Resource Locator, or URL, in the 
Vertical Blanking Interval (VBI) of the video signal. A personal computer equipped 
with an appropriate TV video card and web browser extracts the URL from the VBI 
and, at appropriate times, directs the web browser to retrieve and display the 
identified web page from the Internet. The web page is foreseen for display in a 
portion of the screen within which the video is not displayed. Alternatively, the 
URL is not coded into the VBI but rather is available via a website. In this 
embodiment, broadcasters schedule real live time transmissions of URLs to 
viewers at the same time as the broadcast. This embodiment would also allow 
individual viewers to customize the URL transmissions based on a viewer profile. 

[0012] United States patent No.5,861 ,881 by Freeman et al. discloses an 

interactive computer system. The system is based upon "branches" which occur 
during the course of a full-motion video. The branches may be to another video 
segment, or to graphic and audio segments which are integrated into the video. 
The preferred embodiment uses a combination of branches which are the result 
of viewer input or in response to a trigger point interspersed by the broadcaster 
within the video segment. The system can branch between video from a number 
of sources, e.g. live video broadcasts, video stored on disk and graphic and audio 
data stored on disk. 
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[0013] United States patent No.6,01 8,768 by Ullman et al. describes 

another system for integrating video programming with information available on 
the Internet. URLs, as described above, are imbedded in video transmissions and 
these URLs are used to retrieve web pages related to the transmission. 

[0014] Although the above related sources of prior art allow the viewer to 
receive individualized content and in some manner modify the way in which that 
content is displayed, none of these documents discloses a system which allows 
the viewer to combine a broadcast with graphical elements capable of being 
modified in real-time from any of a number of data sources. Additionally, the prior 
art does not reveal a system where the manner in which data displayed on a video 
screen can be defined by the viewer using a template and graphical elements 
which are positioned within the template. 

SUMMARY OF THE INVENTION 

[0015] The present invention addresses the above limitations by providing 
a system that allows broadcasters to simultaneously deploy interactive content for 
live television, broadband Internet, streaming video, interactive TV, and wireless 
devices. 

[0016] More specifically, according to a first aspect of the present invention, 
there is provided a system for a viewer's customization of the presentation a serial 
transmission and data on a display comprising: 

means for positioning and displaying the serial transmission on the 

display; 

at least one data collector supplying the data; 

at least one viewer selectable graphics object having an 
appearance, wherein each of said at least one graphics object is connected to a 
corresponding one of said at least one data collector and wherein each graphics 
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object appearance is modified according to the data of said corresponding one of 
said at least one data collector; and 

means for positioning and presenting said at least one graphics 
object on the display. 

[0017] According to a second aspect of the present invention, there is 
provided a method for a viewer's customization of the presentation of a serial 
transmission and data on a display comprising: 

positioning and displaying the serial transmission on the display; 

providing at least one graphics object having respective 

appearances; 

selecting at least one graphics object having an appearance; 

positioning said at least one graphics object on the display; 

providing at least one data collector having data; 

connecting said selected at least one graphics object to a 
corresponding one of said at least one data collector; 

transferring the data from said corresponding data collector to said 
at least one graphics object; wherein each graphics object appearance is modified 
according to the data of said corresponding data collector; and 

displaying said at least one graphic object on the display. 

[0018] According to a third aspect of the present invention, there is 
provided a interactive system for modifying graphics displayed with a serial 
transmission comprising: 

means for requesting viewer input; 

means for receiving and processing said viewer input and having an 

output; 

at least one graphics object having an appearance; 
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means for modifying said appearance of said at least one graphics 
object connected to said receiving and processing means, wherein said at least 
one graphics object appearance is modified according to said output; 

means for combining said at least one graphics object with the serial 
transmission to form an enhanced serial transmission. 

[0019] According to a final aspect of the present invention, there is provided 
a method for enhancing a serial transmission and displaying this enhanced 
transmission on a display comprising the steps of: 
in a pre-production phase: 

creating a template including at least one graphics object; 

making the template available; 
in a post production phase: 

providing a serial transmission; 

providing at least one data collector having data; 

acquiring said template; 

linking said at least one graphics object of the template to a 
corresponding data collector; 

modifying said at least one graphics object according to the real time 
modified data of the data collector; and 

displaying the combination of said serial transmission and said at 
least one graphics object on the display. 

BRIEF DESCRIPTION OF THE DRAWINGS 

In the appended drawings: 

[0020] Figure 1 is a schematic diagram of the system in accordance with 

a preferred embodiment of the present invention; 
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[0021] Figure 2 is a schematic diagram of the template authoring station in 
accordance with a preferred embodiment of the present invention; and 

[0022] Figure 3 is a schematic flow chart of the method according to an 

embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0023] Generally stated, the system of the present invention allows for the 

combination of television broadcasting with two-way communication capabilities, 
for example, through of the Internet, to create a customizable and interactive 
television viewing experience. 

[0024] Indeed, the present invention discloses a system for enhancing a 
serial transmission with data. The serial transmission is typically of a type that 
provides moving video images when projected on an appropriate display device. 
The data is typically of a type, as will be seen below, which is complementary to 
the images being displayed but there is essentially no limit to the kinds of data that 
can be displayed. 

[0025] Once combined with the moving images of the serial transmission, 
the data will have been converted into a graphical format. A template is used to 
determine the position of the data converted into its graphical form on the display 
device. The template, which has previously been loaded in the display device, 
provides information to a given display device as to the graphical forms into which 
the data is to be converted and as to the positioning of the data on the display 
device. 

[0026] In a final step, the system combines the data with the serial 

transmission (for example, in the form of moving images) to form an enhanced 
serial transmission in the form of an enhanced signal. 
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[0027] Additionally, the system according to the present invention allows 
for an interactive component to be introduced into a serial transmission by 
providing the means for displaying audience responses to interactive questions. 

[0028] In the preferred embodiment, the serial transmission disclosed is an 
analogue NTSC TV broadcast, although it should be understood to include any 
number of transmission types, including but not limited to streaming video, MPEG 
or digital TV. 

[0029] Referring now to Figure 1 of the appended drawings, a system 
according to an embodiment of the present invention will be described. 

[0030] As is widely known, in a conventional system, a broadcaster 6 
broadcasts a conventional serial transmission to a conventional TV set 54 which 
displays it in a known manner. 

[0031] In the system of the present invention, data is collected and stored 

by a data collector 18. The data is typically of a type with a moderately short 
useful life, and includes information from a variety of different sources including 
online clocks, weather reports, stock and currency exchange reports, sports 
scores and statistics, for example. The data may also be, for example, 
information related to political broadcasts such as polls and vote results, 
candidate biographies, pictures of candidates and related information. Of course, 
the data may be collected in real time, i.e. in synchronism with the serial 
transmission, or before this serial transmission. 

[0032] The preferred embodiment of the system of the present invention 
uses a templates, provided by the broadcaster, to format collected data 16 from 
the data collector 18 which is then combined with a TV broadcast 4 or 4' received 
from the broadcaster 6. Using graphic objects 8, the template, previously loaded 
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in the client's display describes a layout on a display where and how the data 16 
should be displayed on a client's computer 10. 

[0033] The client's display may be a computer 10 equipped to interpret the 
template and receive the enhanced transmission signal or a conventional 
television set 54 provided with an adequate set-top box 55 receiving a signal via 
airwaves or cable, for example. Again, the set top box 55 is configured to 
interpret the template and display the enhanced serial transmission adequately. 

[0034] The graphic objects 8, can be of a number of different types. For 
example, standard user interface graphic objects such as list boxes, drop lists, 
animated buttons and text boxes can be used to display information to a viewer 
or to request viewer input. Additionally, the range of graphic objects 8, including 
their behavior and visual appearance, can be supplemented by specialized 
graphic objects (not shown) which can be developed according to the 
requirements of the producer. This could include, for example, a bar chart, clock, 
pie-chart, archived video or any other graphic or information which is 
complementary to the broadcast. Additionally, graphic objects may be developed 
in ActiveX or Java thereby providing the producer with a virtually unlimited 
opportunity to develop customized graphic objects to be included in a given 
broadcast. 

[0035] A client plug-in application 14 capable of interpreting the template 
2 is installed into the web browser 12 allowing the viewer to load a template 2 and 
use it as a basis for formatting the TV broadcast 4' and selecting and positioning 
graphic objects 8 for displaying data 16 on the web browser 12. 

[0036] The graphic objects 8 receive data 16 from a data collector 18 via 
the Internet 20, for example. The data collectors 18 are the same as those that 
would be used by a broadcaster in creating graphical content for a particular 
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broadcast. Of course, more than one data collector 18 could be used should the 
data be disseminated. Similarly, the connection between the object could be done 
by another medium than the internet, for example, via airwaves. 

[0037] According to a preferred embodiment, the use of the system may be 
divided into a pre-production phase and a post-production phase. 

Pre-production phase 

[0038] The following provides an illustrative example of the pre-production 
phase of a system in accordance with the preferred embodiment of the present 
invention. 

[0039] Referring to Figure 2, in the pre-production phase, a software based 
template authoring tool 22 is implemented on a general purpose computer 
workstation 24, the computer including a display 26, a mouse 28, a keyboard 30 
and an Internet connection 32. 

[0040] Using the template authoring tool 22, the producer selects at least 

one standardized graphic object 8 from a toolbox 34 of available graphic objects 
36 using a mouse 28 or keyboard 30 and inserts the selected graphic object 8 into 
a preferred position on the workspace 38. Note that the available graphic objects 
36 include one or more interactive graphic objects 40 that may also be inserted 
into the workspace 38. The function of these interactive graphic objects will be 
described hereinbelow. The producer also designates the display region 42 for the 
serial transmission within which the television broadcast is to be displayed. Of 
course, the number and type of graphical objects 8 and interactive graphic objects 
40 selected and placed on the workspace 38 is at the producer's discretion. 

[0041] As stated above, the graphic objects 8, can be one of a number of 
different types. 
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[0042] Once the producer has finished selecting and positioning the graphic 

objects 8 on the workspace 38 and selected the video display region 42, the 
workspace 38 may be published to create a template 2. 

[0043] In the preferred embodiment herein described, publishing is 
generally to be understood as rendering the template in a format which can be 
read and understood by an appropriate client plug-in or other pieces of software, 
installed in a viewer's web browser or set top box, for example. The template 2, 
would then be placed, for example, on the broadcaster's web site 44 via the 
Internet connection 32 or supplied to the set top box via the connection (RF or 
cable) to the broadcaster. The viewer would have access to the portion of the 
broadcaster's web site 44 allowing the use of the template 2 as the basis for 
viewing a broadcast provided the viewer's web browser is equipped with the 
appropriate client plug-in. 

[0044] Note that although the above makes reference to the use of the 

broadcaster's website 44 as the preferred method of distribution for the templates 
2 to a viewer, it should be understood that any of a number of conventional 
distribution methods could be used for templates distribution, including CD-ROM 
and floppy disk. 

[0045] A producer may publish a group of templates 46 for each broadcast 
with the viewer able to select the template 2 which corresponds best to the 
viewer's personal tastes. Alternatively, a sole template 2 may be used that 
contains objects that are removable or customizable by the viewer, thereby not 
requiring a user's choice of template. In this case, the user would select which 
graphics object is to be displayed along with the serial transmission. 
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Post-production phase 

[0046] The following provides an illustrative example of the post-production 
phase of the system in accordance with the preferred embodiment of the present 
invention. This example will first describe the operation when the viewer uses a 
computer 10 and then the operation when the viewer uses a conventional TV set 
54. 

Using a computer 

[0047] In the post-production phase, the group of published templates 46 

has been made available to a viewer at an appropriate location, for example the 
broadcaster's website 44, although any conventional storage means could be 
used, including CD-ROM and floppy disk. 

[0048] Referring back to Figure 1, the viewer's computer 10 is equipped 
with an Internet connection 48, for example via a high speed line, a RF connection 
or a conventional modem. A conventionally available web browser is installed on 
the client computer 10 along with the client application plug-in 14 for reading the 
template 2. 

[0049] The viewer selects a single template 2 from the group of templates 
46 on the broadcaster's website 44 for a particular broadcast. In the cases where 
only one template is available, the plug-in may download it automatically without 
the assistance of the viewer. 

[0050] If the template 2 is customizable by the user, the user may do so at 

any time during the post-production phase. 

[0051] On loading the template in the client computer 1 0, the client plug-in 
14 communicates via the Internet connection 48 with a marshaller 50 and 
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requests information regarding the address of the data collector 18. The 
marshaller 48 returns the address of the data collector 18 to the client plug-in 14 
and also indicates whether the connection to be used will be permanent or on 
demand. If permanent, the client plug-in 14 opens a connection to the data 
collector 18 using the TCP connection oriented protocol. If on-demand, the client 
plug-in 14 connects to the data collector 18 via the TCP connection oriented 
protocol and provides the data collector 18 with the address of the client plug-in 
14. The client plug-in 14 then closes the connection and then sets up a local 
server to listen for updates from the data collector 18. Of course, other methods 
could be used to find the address of the data collector 18 or to otherwise link the 
graphics objects with the data collector. 

[0052] The client plug-in 14 receives data 1 6 from the data collector 1 8 from 
time to time. The data 16 will modify the appearance of the graphic object 8 
depending on the graphic object attributes and behavior. There is no requirement 
on behalf of the viewer to refresh the screen for modifications to the graphics to 
take place. Communications between the client plug-in 14 and the data collector 
18 as well as updates to the graphic objects 8 are carried out automatically and 
no interaction on behalf of the viewer is required. 

[0053] Graphic objects 8 may have one or more viewer definable attributes. 
Modification of these attributes may modify general characteristics of the graphic 
object, for example text color or font used, but may also be used to modify the 
general characteristic of the object, for example the display of data as numbers 
or as a bar chart, or modify the behavior of the graphic object 8 on receipt of data 
16 from the data collector 18. One of these attributes is advantageously the 
removal of the object from the displayed objects. 
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[0054] During transmission of the television broadcast 4' interactive actions 
on behalf of the viewing audience may be requested. The interactive graphic 
object 40 provides the interface by which a viewer may provide a response to a 
request for interactive actions, via conventional input devices such as a keyboard, 
a mouse or a remote control, for example. This response may include, for 
example, voting on a particular item or selecting one of a number of pre-defined 
responses displayed by the interactive graphic object 40. When the viewer 
submits a response, the client plug-in 14 transmits, from the interactive graphic 
object 40, response data 58 to the interactive data collector 56. 

[0055] The serial transmission of the enhanced signal 4', that has been 

enhanced by the data, may then be "webcasted" by the broadcaster 6 to the 
computer 10, for example, via the Internet connection 48. 

[0056] The elements required for such webcast are believed well known to 
these skilled in the art and will therefore not be further discussed herein. 

[0057] It is to be noted that should more than one graphic object 8 be used 
on the template 2 selected by the user, the client plug in 14 would potentially open 
two links to two different data collectors 18 (only one shown on Figure 1) should 
the data required be distributed. 

Using a conventional TV set 

[0058] When a conventional TV set 54 provided with a set top box 55 is 
used to obtain and interpret the template2, this template is advantageously 
supplied of the set top box 55 via the connection (for example RF or cable) 
between the box 55 and the broadcaster 6. This template is subsequently used 
to format the incoming enhanced serial transmission as described hereinabove 
so that it may adequately be displayed on the TV set 54. 
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[0059] If the template 2 is customizable by the user, the user may do so at 
any time during the post-production phase, via the use of a remote control, for 
example. 

[0060] When using a set top box, the enhanced serial transmission 
contains the required data supplied by the broadcaster 6 by the data collector 18 
and/or the interactive data collector 56. 

[0061] Again, the data received will modify the appearance of the graphic 
object 8 depending on the graphic object attributes and behavior. 

[0062] During transmission of the television broadcast 4 interactive actions 
on behalf of the viewing audience may be requested. The interactive graphic 
object 40 provides the interface by which a viewer may provide a response to a 
request for interactive actions, via conventional input devices such as a remote 
control, for example. Response data is transmitted to the interactive data collector 
56. 

[0063] Figure 3 of the appended drawings illustrates the above described 
steps of the pre and post production phases. 

[0064] As will easily be understood by one skilled in the art, the broadcaster 
6, the template authoring workstation 24 and the broadcaster website 44 could be 
contained in the same computer or could be distributed in various locations and 
interconnected by a computer network such as, for example, the Internet. 

[0065] Note that although the description of the preferred embodiment 
makes reference to a viewer's web browser and client plug-ins, it should be 
understood that a television equipped with an appropriate software and/or 
hardware capable of selecting and interpreting the templates, or a similarly 
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equipped set-top box, are also contemplated, as are wireless handheld personal 
communication devices equipped with an appropriate interface. 

[0066] Although the present invention has been described hereinabove by 
way of a preferred embodiment thereof, this embodiment can be modified at will, 
within the scope of the appended claims, without departing from the spirit and 
nature of the subject invention. 



